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A related input parameter is used in this case study to forecast solar thermal 
systems (STS) capabilities and to compare which artificial neural network 
(ANN) algorithms and other artificial intelligence (AI) methods have the 
most reliable predictor for STS performance. In order to gauge the 
performance of the STS, this research aims to implement AI for predicting 
STS performance by comparing the ANN technique with other methods. 
Three different training algorithms which are Levenberg-Marquardt (LM), 
scaled conjugate gradient (SCG) and Bayesian regularization (BR) are 
considered in this research. This research will identify acceptable parameters 
and the best AI technique to use in predicting the STS performance. 
Previous research on STS demonstrates that the efficiency of STS has been 
estimated using different input parameters. The results show that the 
prediction of the LM training algorithm is the best for STS performance. 
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1. INTRODUCTION 

Solar thermal systems (STS), which are currently satisfying the growing demand for electricity, are 
efficient and environment-friendly facilities [1]. Thus, methods based on intelligent machines, such as an 
artificial neural network (ANN) and other artificial intelligence (AI) methods, are vital in accurate and rapid 
machine results prediction. Previous researcher that done some analysis on STS reveals that various input 
parameters have been used to forecast the STS output [2]. In addition, researchers have also used a minimum 
number of ANN modelling training algorithms, which are widely used for training Levenberg-Marquardt 
(LM) algorithms [3]-[7]. Due to the research gap on STS, this paper proposes to classify the relevant input 
parameters to predict the performance of the STS and to compare which ANN algorithms and other AI 
methods give the most accurate prediction for STS performance. 


2. SOLAR THERMAL SYSTEMS 

The heat provided by STS may be used for practical purposes such as water or space heating. Unlike 
solar panels, which convert solar energy into electricity, the STS converts sunlight into heat. A solar thermal 
device is a form of solar collector that converts sunlight into heat. To draw sunlight, solar water heating 
systems usually use collectors installed on the roof of a building or home [8]. STS convert sunlight into 
thermal or electrical energy and are used in both commercial and residential buildings [9], [10]. 
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3. ARTIFICIAL INTELLIGENCE TECHNIQUES 
3.1. Artificial neural network 

The cell body serves as a generator, the synapse functions as a reference, and the axon sends feedback 
signals to other neurons and carries out nonlinear operations. Figure 1 specifically shows the components of a 
neuron which is made up of dendrites, a cell body or soma, and the axon. Common biological neurons are 
individual cells, each composed of a major body of cells along with many suckers that extend from that body. 
The body, or soma, houses machinery for maintaining basic cell function and energy processing. There are two 
types of tendrils which are dendrites which receive information from other neurons and carry it to the cell body, 
and axons, which send information from the cell body to other neurons. ANN have been used by a few previous 
researches to represent and forecast STS performance. The use of the ANN technique has been very common in 
the field of thermal engineering systems in the last two decades [11]-[18]. 
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Figure 1. Basic structure of biological neurons 


3.2. Type of artificial neural network training algorithm 

In terms of memory requirements, processor speed, and numerical precision, all the parameters have 
different characteristics and performances. In a neural network, the method used to perform a learning 
process is called an optimisation algorithm. It is very challenging to establish which training algorithm is the 
quickest and optimal for a given problem [19]. 


3.2.1. Levenberg-Marquardt training algorithm 

The teaching method of LM is derived from the Newton method [20]. The algorithm has a quicker 
integration of second orders, fewer iterations, and the Hessian matrix need not be computed. For certain 
network models that have a small number of parameters, the algorithm has a faster training speed. Let N be 
the weight and bias vector of each layer in the iterative training provided by (1). Delta N is the change in N, 
and changing N involves controlling the weights and thresholds of each layer in the network; eventually, the 
objective of network training is accomplished [21]. 
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Where, the expression function of P(N)=e;(N)?, i=/—I/ represent the square of error. J(N) is the Jacobian 
matrix given by (3), where E(N) is expressed as e; (N), i=/—>/ constituent vectors, and S(N) is the error 
function given by (4). Since the LM algorithm refines the Gauss-Newton process, AN can be represented by (5). 
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Where, u > 0 is a constant and I is the unit matrix. 

If u=0 is used for the Gauss-Newton method, LM follows the small-step gradient descent (GD) 
method, as it is larger. The adjustment factor is changed from x to a. u is increased or decreased if the 
training fails. Due to its positive value, the solution of AN still remains. The LM algorithm is equivalent to 
the system of Gauss-Newton. The measures of the LM algorithm can be described by the following steps: 

— Initialise the permissible values, coefficients, thresholds, error training weights, and k=0 (k is the number 
of iterations). 

— Calculate the network output P(N) and Jacobian matrix J(N) expression functions using (2) and (3), 
respectively. 

— Using (5) to calculate AN. 

— The simulations are terminated if P(N)<e, otherwise, use N+4 as the weights and thresholds to recalculate 
the function of expression P(N). Let 2=//a and p=p+1 and return to 2) when P(N) is less than the P(N) of 
2); otherwise, set “= and return to 3). 


3.2.2. Scaled conjugate gradient training algorithm 
For significant problems, the scaled conjugate gradient (SCG) training method can be useful. It 
utilises second-order knowledge sans the process of line-search. The level of memory consumption can, 
therefore, be minimised by cutting down the volume of computed gradient information [22]. 
— Select the weight vector Ww, and scalars OS o = 10-40 4 < 1076,4 = 0. 
Let ġ = 7, = —E'(W,), p = 1 and success = true. 
— If success=true, calculate the second order using (6), (7), and (8). 


oO 
T- (6) 
= [E(w +0pBp)-E' (Wy )] 
Š,- 5 (7) 
ôp = Pp Sp (8) 


— If the steepest direction of descent is 7, # 0, set p = p+1 and go to 2); otherwise, terminate and return 
Wp+1 as the minimum required. 


3.2.3. Bayesian regularization training algorithm 

The algorithm operates effectively and illustrates greater efficacy in preventing limited local issues 
through extraction of weights from the network. A robust estimation of noise and complexity is given by the 
Bayesian regularization (BR) algorithm. In the BR algorithm, cross-validation is unnecessary for validation 
purposes, which avoids part of the training results. 

The E(w) gradient and (9) function are considered the first derivatives of the error function. The LM 
algorithm developed by Kenneth Levenberg and Donald Marquardt should be used by ANNs that deal with 
modest-sized problems. The second E(w) derivative is the Hessian of E(w) in (10). Wilamowski and Yu 
introduced the Hessian approximation, as demonstrated in (11). The LM algorithm update rule is provided as 
(12) [23]. 
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Where J: Jacobian matrix (matrix of first derivatives with respect to weight vector), u: combination 
coefficient, and Z: identity matrix. 


3.3. Linear regression 

Regression analysis is a mathematical method in which the association between variables is 
investigated and modelled. The basic linear regression model is a model that has one independent variable, x, 
which is related to one answer variable, y, which is a straight line. This simple linear regression model is 
given by (13). 


y= Pot bix + € (13) 


Where an undefined constant is the intercept, o, and the slope, 6,, and a random error is e. It is assumed that 
the errors have a mean of zero and an undefined variance, o°. The By and $, parameters are undefined and 
must be calculated using sample data. The basic theorem of linear regression is sometimes called the Least 
Squares Regression equation. This shows the criteria used to select the best fitting rows, in which the number 
of residual squares should be the least. As such, as shown in (14), the least squares regression is the line for 
which the sum of squared residuals is minimum [24]. 


Lehi -y (14) 


3.4. Support vector machine 

In 1992, Vapnik first implemented the support vector machine (SVM) as a learning concept for 
classifying pattern recognition. Capacity analysis places SVM as the latest method in pattern recognition in 
numerous implementations and it is a theme that is growing rapidly in popularity. SVM is a learning machine 
method that functions based on the structural risk minimisation (SRM) principle [25]-[29]. SVM also 
provides improved classification outcomes and are commonly used for methods of pattern detection, such as 
optimum probability and classifiers of the neural network. Radial basis function and multilayer perceptron 
classifiers resolve quadratic programming (QP). 

The learning problem in SVM setting is as follows: between some high-dimensional input vector, x, 
and scalar output, y; there is some undefined and nonlinear dependence (mapping, function) y=f(x) (or the 
vector output, y, as in the case of multiclass SVM [30]. The structure of the optimised SVM is shown in 
Figure 2. In SVM, using nonlinear mapping, the historical data of the time series is transformed into higher- 
dimensional space [31]-[33]. 
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Figure 2. Structure of the optimised SVM 


4. METHOD 

In this paper, ANN will be designed to obtain the efficiency of the STS. MATLAB software with 
the aid of certain techniques and applications will be utilised. Additionally, to validate the proposed 
technique, other AI methods will also be developed. 


4.1. Flowchart and block diagram of the project 

The ultimate process of ANN prediction is shown in Figure 3(a). Other than that, training 
performance for various ANN training algorithms and training percentages is also part of the process. In the 
beginning of the process, data are gathered and undergo normalisation. Then, the selected data undergo 
training and testing process. The simulation will display the prediction results upon no testing error. 

Figure 3(b) shows the machine learning flowchart for the LR and SVM methods used to predict the 
performance of STS. Initially, the training data are normalised before some features are extracted, resulting in 
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the machine learning model, which are LR and SVM. Next, the prediction result is compared with actual data 
to establish the error by quality matric. 
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Figure 3. Block diagram of (a) artificial neural network prediction process and (b) machine learning 
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Figure 4(a) shows that the ANN block diagram consists of two input layers with 10 hidden layers and 
one output layer for LM, SCG, and BR training algorithm. Figure 4(b) shows the linear regression and SVM 
block diagram, which consists of one input and output with five-fold validation. This research used one input 
parameter, which is temperature, while the output that needs to be predicted is relative humidity (%). The fold 
validation is set to five to provide optimum prediction output. If fold validation is set at more than five, the 


difference in size between the training set becomes smaller and will affect the prediction performance. 


Figure 4. Block diagram of (a) ANN method and (b) linear regression and SVM methods 


4.2. Normalised data 
Every prediction conducted in this research had utilised normalised data in order to give an optimal 

result with the range of normalisation 0 to 1. The data are taken from an article by Anastasia Mylona, which 

is modelling the thermal performance of complex glazing systems [34]. In (15) is used to normalise the data. 
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Where Xi=normalised data, Xni=data that need to be normalised, Min (Xn)=minimum value in data set, and 
Max (Xn)=maximum value in data set. 


4.3. Software development 

The MATLAB software was used in the whole process of prediction, in which the MATLAB 2019b 
version was used to create programming based on the project. The MATLAB software has two media that 
were used in this project, which are programming script and application tools such as neural net fitting tool 


(nftool) and regression learner application. 
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4.3.1. Neural net fitting tool 
The nftool shown in Figure 5 is used to add 
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ress the issue of data fitting, used together with LM, BR, 


and SCG to train the two-layer feed-forward network [35]. By using the nftool, the neural fitting will assist in 
selecting data, create and train a network. It also assists in the evaluation of its performance using mean 


square error and regression technique. 


@ Neural Fitting (nftool) 


Welcome to the Neural Network Fitting app. 


Solve an input-output fitting problem with a two-layer feed-forward neural network. 


Introduction 


In fitting problems, you want a neural network to map between a data set 
of numeric inputs and a set of numeric targets. 


Examples of this type of problem include estimating engine emission 
levels based on measurements of fuel consumption and speed 

or predicting a patient's bodyfat level based on body 
measurements . 


The Neural Fitting app will help you select data, create and train a network, 
and evaluate its performance using mean square error and regression 
analysis. 


» To continue, click [Next]. 


© Neural Network Start Hd Welcome 


Neural Network 


A two-layer feed-forward network with sigmoid hidden neurons and linear 
output neurons , can fit multi-dimensional mapping problems 
arbitrarily well, given consistent data and enough neurons in its hidden 
layer. 


The network will be trained with Levenberg-Marquardt backpropagation 


algorithm , unless there is not enough memory, in which case 
scaled conjugate gradient backpropagation will be used. 


@ Back Q Cancel 


Figure 5. 


4.3.2. Regression learner application 
Figure 6 shows that MATLAB used the 


Neural nftool 


regression learner application to help select between 


different algorithms to train and test regression models, compare their validation errors side-by-side after 
practicing several models, and select the better prediction performance in terms of mean squared error 
(MSE), root-mean-square error (RMSE), and training time. It also assists in deciding which algorithm to use. 
In regression learner application, Figure 7 displays the flow map for training regression models [36]. 
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Figure 6. Regression learner application 
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Figure 7. Flowchart of training regression models 


5. RESULTS AND DISCUSSION 

The prediction result is divided into three sections: first, the comparative prediction performance of 
STS with the ANN training algorithms, such as LM, SCG, and BR. The second section compares the 
prediction performance of STS with ANN training algorithm and different training percentages, in which the 
training percentage was increased from 50% to 70% for all ANN training algorithms: LM, SCG, and BR. 
Finally, a comparison was made on the prediction performance of STS with artificial neural network (ANN) 
and other AI methods, such as LR and SVM. 


5.1. Simulation outputs 
5.1.1. Comparative study of artificial neural network with different training algorithm 

Based on Table 1, the LM training algorithm shows the best regression value of almost 1, which is 
at 0.99. At the same time, the algorithm training time is faster compared with BR and SCG. BR training 
algorithm shows that the regression value is almost like that of LM. 


Table 1. Comparison of errors, regression coefficient, and training time by different ANN algorithms 
Training algorithm _ RMSE (% of RH) MSE (% of RH) _ Regression coefficient (R*) Time elapsed (s) 


LM 0.0011 0.000064702 0.999 0.41648 
BR 0.00080072 NaN 0.99939 1.395452 
SCG 0.0049 0.0008278 0.97613 0.420273 


Note: MSE for BR training algorithm cannot be calculated using the software 


5.1.2. Comparative study of artificial neural network training algorithm with different training 
percentage 

According to Table 2, the LM training algorithm with 70% training percentage is the best prediction 
with regression value 0.999. The RMSE for 70% training percentage is considered the least value of RMSE, 
which is 0.0011. Meanwhile, 50% training percentage has the regression value of 0.9985 with an RMSE 
value of 0.0013. The training time for 70% training percentage is longer than that of 50% training percentage, 
where the LM with 70% training percentage gave the best prediction results. Finally, the 70% of SCG 
training algorithm is deemed the best prediction due to its regression value of 0.97613, which is close to 1, as 
compared to that given by the 50% training percentage. Overall, in this section, the LM training algorithm 
with 70% training percentage provided the best prediction compared with others. 


Table 2. Comparison of errors, regression coefficient, and training time by different training percentages 


Training algorithm Piha, %) ( ae MSE (% of RH) Pen Time elapsed (s) 
LM 50 0.0013 0.00029229 0.99856 0.390622 
70 0.0011 0.000064702 0.999 0.41648 
BR 50 0.00079409 NaN 0.99941 0.861464 
70 0.00080072 NaN 0.99939 1.395452 
SCG 50 0.0063 0.0052 0.96349 0.420051 
70 0.0049 0.0008278 0.97613 0.420273 


5.1.3. Comparative study of artificial neural network with other artificial intelligence methods 

Table 3 the best prediction in this section is ANN, with a regression value of 0.999. The values of 
both errors, RMSE and MSE, are lower than that of other methods, which are linear regression and SVM; in 
which their training time is faster compared with others. If comparison is made between linear regression and 
SVM, it will demonstrate that SVM provides the best prediction even with longer training time compared 
with linear regression. 

As shown in Figures 8(a) and 8(b), the linear regression plot errors are more significant, causing the 
lower accuracy of prediction, while in SVM, the error is least and thus, the accuracy of prediction is higher. 
Figures 9(a) and 9(b) show the comparison plot between the actual value and predicted value for linear 
regression and SVM, respectively; the blue colour is the expected value. 
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Table 3. Comparison of errors, regression coefficient, and training time by different artificialintelligence methods 
Training algorithm RMSE (% of RH) _MSE(% of RH) Regression coefficient (R*) Time elapsed (s) 


Linear regression 0.0031733 0.001007 0.98 0.79793 
SVM 0.0015825 0.00025044 0.99 4.1361 
ANN 0.0011 0.000064702 0.999 0.41648 
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Figure 8. Response plot of (a) linear regression and (b) SVM 
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Figure 9. Predicted vs actual plot of (a) linear regression and (b) SVM 


6. CONCLUSION AND RECOMMENDATION 

This work concerns the application of AI for STS prediction performance. This analysis 
demonstrated one of the better training algorithms, which typically had a value similar to the actual value 
from the predicted training algorithm of LM. In the second section, the contrast with a training percentage 
between the training algorithm of ANN shows that the percentage of preparation has influenced the 
prediction results. The better approaches of AI from this section are the ANN relative to other techniques, 
which are linear regression and SVM. The ANN produces superior results in the prediction of STS 
performance. Only then can further analyses of different hidden layers of every training algorithm can be 
developed in the future. 
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